Engineers use compression (also called coding or encoding) to reduce the bit rate of digital video. Compression decreases the cost of storing and transmitting video by converting the video into a lower bit rate form. Decompression (also called decoding) reconstructs a version of the original video from the compressed form. A “codec” is an encoder/decoder system.
When it converts video to a lower bit rate form, a video encoder can decrease the quality of the compressed video to reduce bit rate. By selectively removing detail in the video, the encoder makes the video simpler and easier to compress, but the compressed video is less faithful to the original video. Aside from this basic quality/bit rate tradeoff, the bit rate of the video depends on the content (e.g., complexity) of the video and the format of the video.
Video information is organized according to different formats for different devices and applications. Attributes of video format can include color space, chroma sampling rate, sample depth, spatial resolution and temporal resolution. Typically, quality and bit rate vary directly for spatial resolution (e.g., detail in a picture) and temporal resolution (e.g., number of pictures per second), with higher spatial resolution or higher temporal resolution resulting in higher quality but also resulting in higher bit rate.
In video encoding and decoding applications, common color spaces include YUV and YCbCr. Y indicates the brightness (luma) component of the video, while U and V, or Cb and Cr, indicate color (chroma) components of the video. Aside from YUV and YCbCr, many other color spaces organize video in a luma channel and chroma channels.
Chroma sampling rate refers to sampling of the chroma channels of video relative to the luma channel. In the YUV color space, for example, one chroma sampling rate is 4:4:4, which indicates that for every Y sample, corresponding U and V samples are present. The human eye is more sensitive to variations in brightness than color, however, and encoders have been developed to take advantage of this fact. Another chroma sampling rate is 4:2:2, which indicates that a single U sample and a single V sample correspond to two horizontal Y samples. Chroma sampling rates at lower resolution, such as 4:2:2 or 4:2:0, result in fewer samples and typically require fewer bits to encode than higher resolution chroma sample rates, such as 4:4:4. Because of the prevalence of 4:2:0 chroma sampling, certain video encoders accept video in 4:2:0 format, but not source formats with higher chroma resolution.
Each picture element (“pixel”) of a video picture includes one or more samples, and each sample is digitally represented with one or more bits. Studios and content producers often use video with 10 bits per sample or 12 bits per sample to represent sample values more precisely, with more gradations of brightness or color. Using higher sample depth allows for greater precision in sample values, or allows for capturing a wider color gamut. For example, 12-bit sample values have more possible values than 10-bit sample values or 8-bit sample values. As a tradeoff for this higher quality, higher sample depth tends to increase bit rate for encoding and decoding applications. Many encoders accept video with 8-bit samples, according to convention.
Scalable video encoding and decoding facilitate delivery of video to devices with different capabilities. A typical scalable video encoder splits video into a base layer and one or more enhancement layers. The base layer alone provides the video for reconstruction at a lower resolution, and the enhancement layer(s) can be added to provide extra information that will increase video quality. Some scalable encoders and decoders rely on temporal scalability of video. Other common scalable encoding/decoding schemes involve scalability for either the spatial resolution or the overall encoding quality of the video.
Scalable video codecs that support temporal scalability, spatial scalability and/or overall encoding quality scalability provide many options for base and enhancement layers. While these types of scalability provide acceptable performance in many scenarios, they do not have the benefits and advantages of the techniques and tools described below.